A conventional audio-video synchronous playback apparatus reads digital compressed audio data and digital compressed video data from a CD-ROM or a hard disc unit, expands them, and plays back the expanded audio data through a speakers and the expanded video data through a display device such as a CRT monitor.
Conventional audio-video synchronous playback apparatuses for expanding digital compressed audio data and digital compressed video data, respectively, and for synchronously playing back the expanded audio data and the expanded video data (hereinafter, this apparatus will be referred to as the "playback apparatus") are divided broadly into two categories: the first category includes a playback apparatus which has an enough data throughput capacity or which handles a small-sized compressed codes of video data, and the second category includes a playback apparatus which has an insufficient data throughput capacity or which handles a large-sized compressed codes of video data. With the playback apparatus in the first category, all audio data and video frames are played back without interruption because the expansion processes for the audio data and for the video data are performed rapidly. Meanwhile, with the playback apparatus in the second category, the video portion may be played back in slow motion and the audio is intermittently played back, not in a real time manner, because the expansion processes is performed slowly.
The audio which has been intermittently played back may often cause incompatibility to human senses, and the process for the audio data is therefore prior to the process for the video data so that the audio is continuously played back. The conventional apparatus assigns the portion of the throughput capacity, which has not been assigned to the expansion and playback processes for the audio data, to the expansion and playback processes for the video data. When synchronization (coinciding of playback current positions) between the audio data and the video data is not established, a time lag may occur between the audio portion and the video portion, and an appropriate thinning process for the video frames (dropping of the video frames) has been therefore necessary.
One known standard for digital compression is MPEG (Moving Picture Experts Group), where multiplexed, compressed, and coded audio-video data are synchronously played back. In a compression process, time information for executing playback and display are attached to the audio data and the video data. In the expansion process, the playback is performed synchronizing the audio data with the video data with reference to the time information.
As is conventionally known, various types of synchronous playback methods for the audio data and the video data have been proposed. For example, one conventional method is disclosed in "An Image and Audio Decoder" in Japanese Patent Application, First Publication No. Hei 7-75059 (hereinafter, it is referred to as "Citation 1"). When audio playback precedes video playback on a time basis, the image and audio decoder interpolates the audio data in every block calculated based on a time delay, thus synchronizing the audio data with the video data. Further, when the video playback precedes the audio playback on a time basis, the image audio decoder continues displaying of the video portion for the delay time, thus synchronizing the audio data with the video data.
Another method is disclosed in "An Animation Sound Synchronous Control Apparatus" in Japanese Patent Application, First Publication No. Hei 7-110756 (hereinafter, it is referred to as "Citation 2"). The animation sound synchronous control apparatus sets the next images to be displayed based on a playback start time, a display process completion time, and the number of frames to be displayed per unit time (frame rate), so that video playback in accordance with the frame rate can be provided. The animation sound synchronous control apparatus plays back data accompanied by the video portion in audio playback, thereby synchronizing audio data with the video data.
However, the problem arises that, for example, when data having audio data which include background music and video data include important data, there is a possibility that necessary frames may be dropped through synchronization of the audio data with the video data by continuously dropping the video frames in the conventional method.
According to the Citation 1, because the display or the playback is performed in accordance with whichever is delayed, the playback may not be performed in a real-time manner, because the video portion may be played back in slow motion.
While in the Citation 2 the frame rate is adjusted by skipping some video data during playback, some audio data are also skipped when the video data are skipped over because the audio data accompanies the video data. The audio portion may be intermittently played back under an environment where a CPU has low capacity.